Vehicular Telemetry systems are known in the prior art where a vehicle may be equipped with a vehicular telemetry hardware device to monitor and log a range of vehicle parameters. An example of such a device is a Geotab™ GO device. The Geotab GO device interfaces to the vehicle through an on-board diagnostics (OBD) port to gain access to the vehicle network and engine control unit. Once interfaced and operational, the Geotab GO device monitors the vehicle bus and creates of log of raw vehicle data. The Geotab GO device may be further enhanced through a Geotab I/O expander to access and monitor other variables, sensors and devices resulting in a more complex and larger log of raw data. Additionally, the Geotab GO device may further include a GPS capability for tracking and logging raw GPS data. The Geotab GO device may also include an accelerometer for monitoring and logging raw accelerometer data. The real time operation of a plurality of Geotab GO devices produces and communicates multiple complex logs of some or all of this combined raw data to a remote site for subsequent analysis.
The data is considered to be big telematics data due to the complexity of the raw data, the velocity of the raw data, the variety of the raw data, the variability of the raw data and the significant volume of raw data that is communicated to a remote site on a timely basis. For example, on 10 Dec. 2014 there were approximately 250,000 Geotab GO devices in active operation monitoring, tracking and communicating multiple complex logs of raw telematics big data to a Geotab data center. The volume of raw telematics big data in a single day exceeded 300 million records and more than 40 GB of raw telematics big data.
The past approach for transforming the big telematics raw data into a format for use with a SQL database and corresponding analytics process was to delay and copy each full day of big telematics raw data to a separate database where the big telematics raw data could be processed and decoded into a format that could provide meaningful value in an analytics process. This past approach is resource consuming and is typically run during the night when the number of active Geotab GO devices is at a minimum. In this example, the processing and decoding of the big telematics raw data required more that 12 hours for each day of big telematics raw data. The analytics process and corresponding useful information to fleet managers performing fleet management activities is at least 1.5 days old, negatively influencing any real time sensitive fleet management decisions.